Summary:
Hand gesture recognition and classification play a pivotal role in automating Human-Computer Interaction (HCI) and have garnered substantial attention in research. In this study, the focus is placed on the application of gesture recognition in surgical settings to provide valuable feedback during medical training. A tool gesture classification system based on Deep Learning (DL) techniques is proposed, specifically employing a Long Short Term Memory (LSTM)-based model with an attention mechanism. The research is structured in three key stages: data pre-processing to eliminate outliers and smooth trajectories, addressing noise from surgical instrument data acquisition; data augmentation to overcome data scarcity by generating new trajectories through controlled spatial transformations; and the implementation and evaluation of the DL-based classification strategy. The dataset used includes recordings from ten participants with varying surgical experience, covering three types of trajectories and involving both right and left arms. The proposed classifier, combined with the data augmentation strategy, is assessed for its effectiveness in classifying all acquired gestures. The performance of the proposed model is evaluated against other DL-based methodologies commonly employed in surgical gesture classification. The results indicate that the proposed approach outperforms these benchmark methods, achieving higher classification accuracy and robustness in distinguishing diverse surgical gestures.
Spanish layman's summary:
Este estudio se centra en el uso del reconocimiento de gestos en la formación quirúrgica, proponiendo una red neuronal LSTM con un modelo de atención. El enfoque, mejorado con aumento de datos, supera a otros métodos de Deep Learning en la clasificación de gestos quirúrgicos.
English layman's summary:
This study focuses on using gesture recognition in surgical training, proposing a Deep Learning (DL) system using an LSTM-based model with attention. The model, enhanced with data augmentation, outperforms other DL methods in classifying surgical gestures.
Keywords: attention-based LSTM neural networks, data augmentation, deep learning, gestureclassification, surgical gestures
JCR Impact Factor and WoS quartile: 3,000 - Q2 (2023)
DOI reference:
https://doi.org/10.1111/exsy.13706
Published on paper: December 2024.
Published on-line: September 2024.
Citation:
S. Sorbet Santiago, J. Cifuentes, Deep learning-based gesture recognition for surgical applications: A data augmentation approach. Expert Systems. Vol. 41, nº. 12, pp. e13706-1 - e13706-21, December 2024. [Online: September 2024]
